People love to see imagery all around them. Size, brightness, resolution, contrast ratio, 3D and many other features attract the attention of viewers. The goal in creating a display system is to create the best experience possible for the viewer. Creating the best experience often means optimizing the quality of the display. Quality factors include, but are not limited to, geometric accuracy, color accuracy, contrast, resolution, and freedom from distracting artifacts and other performance properties which contribute to the generally pleasing nature of the image. These factors may also include allowing the displayed digital image to accurately represent the original digital image or an image found in nature. To achieve the best possible experience for the user, and/or quality of the display, it is desirable to correct for certain inaccuracies in the image produced by a display by applying corrective measures to image information and tuning the operating point of the display system.
Display systems are composed of one or more display units. Display units may be a variety of types, including, but not limited to, flat panel displays, projectors, emissive displays, e-ink displays, etc. They may be flat or curved. Examples of such displays are listed in commonly assigned U.S. Published Patent Application No. 2008/0246781 A1, entitled SYSTEM AND METHOD FOR PROVIDING IMPROVED DISPLAY QUALITY BY DISPLAY ADJUSTMENT AND IMAGE PROCESSING USING OPTICAL FEEDBACK (claiming the benefit of U.S. Provisional Application Ser. No. 60/895,070) and U.S. patent application Ser. No. 12/818,102, entitled SYSTEM AND METHOD FOR INJECTION OF MAPPING FUNCTIONS whose disclosures are included here by reference by way of useful background information. Each of these types of display unit may suffer from different artifacts.
Flat panel type displays, for example, often suffer from color and intensity sheen within panels, and color and intensity differences across panels. They may also suffer from different input-output curves. For example, two or more discrete displays can show the low intensity gray levels very similarly, but the high intensity gray levels may vary widely from panel to panel. Problematic geometric issues may also arise due to occlusions by panel bezels, misalignment of multiple-panels, the desire for unusual display shapes within a panel, panels being aligned in shapes such as a cylinder, etc.
Projection-based displays suffer from geometric distortions, sometimes on a per-color channel, often as a result of imperfect optics in the projectors. They also suffer from intensity variations within and across projectors, color sheens, color mismatches across projectors, varying black levels, different input-output curves, etc.,
For the display of 3D images, often a different image is respectively presented to the right eye and the left eye. Methods for accomplishing this can include using time to alternate images delivered to each eye, using properties of the light such as polarization or wavelength to select which eye will receive a particular image, using optics to attempt to deliver a different image to each eye based on the eye's spatial location, etc. For 3D images, as with standard images, there may be geometric artifacts, color and intensity artifacts, and potentially different artifacts for the image intended for each eye.
Corrections made to the system can occur at many stages in the chain to displaying the image. One example of a correction can occur in the creation of the digital signal, such as described in detail in the above-incorporated SYSTEM AND METHOD FOR INJECTION OF MAPPING FUNCTIONS. One particular example is shown and described relative to the projector or intermediate warping boxes such as the OMTE parameters, as described in detail in the above-incorporated U.S. Patent Application entitled SYSTEM AND METHOD FOR PROVIDING IMPROVED DISPLAY QUALITY BY DISPLAY ADJUSTMENT AND IMAGE PROCESSING USING OPTICAL FEEDBACK.
The types of corrections generally contemplated herein include warping imagery onto the screen. If using projectors, one correction entails blending imagery across projectors so that the total intensity of a region overlapped by multiple projectors is similar intensity to the rest of the display. Corrections for color and intensity changes both across display units and within display units are also contemplated. This discussion focuses on geometrical changes. Many types of imagery could be shown on the scene. Sometimes the content is effectively a rectangular image that may be blended and warped onto the screen. Sometimes the content consists of many views of a three dimensional scene, where potentially each display unit may be given a different view of a three dimensional scene, and each display units stretches and warps the views so that the resulting display system appears to show one very large view of the entire system. In this case, the content is often rendered using a three-dimensional rendering engine like OpenGL or DirectX.
Manual geometric calibration is the process of having a user specify the geometric correction (the warp) of imagery intended to be shown on a display unit. For manual geometric calibration, the state of the art generally involves the application of substantial precision, which is often tedious to achieve.
The most common method for manual calibration of projectors in a conventional implementation occurs with perfect alignment on a flat screen. For the case of a 1×N array of projectors, for example, these projectors are set up so that the projected images have no relative vertical shift, and a known horizontal shift. In this case, the mapping from input imagery to the screen is typically very straightforward. For a horizontal setup, the left portion of the image goes to the left-projector; the next projector shows some of the same pixels (for the overlap region) and then the pixels directly to the right. The mapping to the screen of the input imagery is therefore an identity warp, with portions of the imagery distributed to each projector. In this case, the precision and tediousness comes in two places. First, the projectors must be aligned—a challenging and difficult process. Second, the transfer function from the projectors to the screen must be very close to a scale mapping. To achieve this degree of precision in the mappings often requires projectors with expensive optics.
If the projectors are not precisely aligned, or the mapping is not a perfect scale factor, a second method of manual calibration involves moving a significant number of points to indicate the correction that should be applied to each projector. Often the points are selected with a pointing device such as a mouse. Two standard examples include the use of applications such as Watchout™ (available from Dataton of Linkoping Sweden), and the eWarpDesigner™ (available from Flexible Picture Systems, Richmond Hill, Ontario, Canada). In each of these exemplary applications, the user has a grid of points that start as evenly spaced across the projector. They are moved to indicate where an image that is sent to the projector should actually be shown on that projector; that is, the points indicate a warp to the input image that is to be projected on the screen. Often, something of the order of a 7 by 9 grid of points (as is often done in the eWarpDesigner), 63 points per projector, must be moved and generally moved precisely. The work is tedious. Often, to be accomplished accurately, the screen is pre-measured and small dots are placed on the screen so that users can move points on the projectors to the correct place on the screen, so that an image will be mapped to the screen correctly. Precision is often required, particularly in an overlap region where small misalignments can become noticeable as a shadowing effect when the final imagery is displayed. Overall, lack of precision can result in artifacts such as an image moving across the display system shrinking and expanding as it moves, or oscillating up and down slightly.
Another example of manual calibration is the application from 3D Perception (Norway). In the software that ships with their compactUTM warping boxes, the user may have a model of where each projector is in 3D space, how it projects its image, and a model of the screen. The starting point can be very helpful, as there is an initial estimate of the warp for the system, but once again, in order to refine the model to achieve good alignment between the projectors, many points on each projector must be moved separately to modify the warps to match from one projector to another.
These kinds of manual methods for modifying and updating warps typically do not use the shape of the screen, or the fact that display units can be modeled—such as a projector as a 4×3 projection matrix with radial distortion, for example. And, they do not use any algorithms that form constraints between the projectors. Effectively, these methods do not use mathematical tools like those used in machine vision that have been created to effectively model 3D scenes, and objects whose geometric properties are dominated by optical principles. Furthermore, these methods typically attempt to warp each projector independently to the screen. They do not try and map projectors to a common coordinate system using points selected in the overlap region, and then take that common coordinate system and warp it to the screen. The goal of this patent is to use the information just described to reduce the tediousness and precision that is often required in a geometric manual calibration of a system.